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DETAILED ACTION 
Remarks 

1 . The finality of the Office action dated 12/29/05 has been withdrawn based on 
Applicant's arguments dated 4/1 1/06. Claims 1-60 have been examined and rejected. 
This Office action is responsive to the amendment filed on 4/1 1/06, whidh has been 
entered in the above identified application. 



Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obviousiat the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



3. Claims 1-4, 7-9, 12-16, 18-21, 24-26, 29-33, 35-38, 41-43, 45-46, 48-51, 54-56, 
and 58-59 are rejected under 35 U.S.C. 103(a) as being unpatentable over Paroz et al 
(U.S. Patent No. 6,587,125), Bickmore et al (U.S. Patent No. 6,857,102), and Fagioli 
(U.S. Patent No. 6,710,790). 



3-1 . Regarding claims 1,18, 35, and 48, Paroz teaches the claim of a remote 
operation system for remotely operating a device to be operated through a 
communication network from an operation side terminal, by disclosing a method for 
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remotely controlling a first computing device from at least one of a plurality of second 
computing devices [column 3, lines 49-67; column 4, lines 1-2]. 

Paroz teaches screen analysis means for analyzing information displayed on a 
screen of said device to be operated, by disclosing that static and dynamic logic of the 
first computing device's user interface is analyzed and used to send an equivalent user 
interface to the second computing device [column 3, lines 56-62] ' 

Paroz teaches screen information transmission means for transmitting data of an 
analysis result obtained by said screen analysis means to said operation side terminal 
to display the data on said operation side terminal, by disclosing that a mediator sends 
the second computing device a set of DHTML or WML pages, which run on the second 
computing device's Web browser. These pages form the second user interface by which 
user can control the first computing device [column 8, lines 19-23], 

Paroz teaches wherein said screen analysis means determines whether or not 
an active window is present on the screen of said device to be operated, and if so, said 
screen analysis means obtains property information of objects displayed on said active 
window, by disclosing that a visual status monitor monitors GUI events generated by the 
local program and updates the widgets running on the second computing device 
[column 8 } lines 66-67; column 9, lines 1-2] Thus, any window that is a6tive will be 
analyzed and displayed. 

Paroz further teaches that a web page for each layout (window) in the GUI is 
created [column 6, glossary; column 8 } lines 51-56], The web pages can be customized 
for properties such as communication protocol (WAP) and parameters for a specific 



Application/Control Number: 09/910,778 
Art Unit: 2173 



Page 4 



computing device such as a cell phone [column 8, lines 57-65], This would allow the first 
computer to be remotely controlled by a mobile device. Paroz does not expressly teach 
wherein said screen information transmission means transmits only the-data of the 
active window and the objects displayed therein, to said operation side terminal, and 
does not transmit any information on the screen of said device to be operated that does 
not correspond to the active window. Bickmore teaches providing devices with limited 
communications bandwidth and small displays to have access to the Internet using re- 
authoring and filtering systems [column 4, lines 65-67; column 5, lines 1-4]. As 
disclosed by Bickmore, cell phone display areas are too small to display web pages 
designed for display on color monitors [column 1, lines 38-53]. Bickmore discloses a 
solution by using web page filtering to allow a user to see only a single web page and 
only those portions of a page that the user is interested in. This would conserve wireless 
bandwidth and device memory [column 3, lines 21-30]. It would have been obvious to 
one of ordinary skill in the art at the time the invention was made, to include in the 
remote control system of Paroz, only viewing a single web page at a time when 
navigating web pages as taught by Bickmore. This would conserve wireless bandwidth 
and device memory. 

Bickmore further teaches that the document re-authoring system; can be 
implemented on a programmed general purpose computer as well as oh a special 
purpose computer, a programmed microprocessor or microcontroller and peripheral 
integrated circuit elements, an ASIC or other integrated circuit, a digital -signal 
processor, a hardwired electronic or logic circuit such as a discrete element circuit, a 
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programmable logic device such as PLD, PLA, FPGA, or PAL, or the like. In general, 
any device capable of implementing a finite state machine, can be used to implement 
the automatic document re-authoring system [column 27, lines 26-48]. \\ would have 
been obvious for one of ordinary skill in the art to implement the re-authoring system on 
the device to be operated in order to reduce the processing load on the client device 
and eliminate the need for an additional device and additional overhead that may be 
needed. 

Fagioli teaches tracking the active window of a host computer in a remote 
computer display window. The viewport which determines which portion of the host 
computer display image is reproduced within the remote application display window can 
automatically be adjusted so as to display the portion of the host computer display 
image which includes the currently active host window [column 2, lines 29-35]. This is 
useful in situations where it is not possible to duplicate the entire host computer screen 
on the remote computer [column 1, lines 36-44], This invention identifies the part of the 
display the user would be most interested in seeing, i.e. the active window. 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made, to include in the remote control system of Paroz and Bickmore, 
tracking the active window of a host computer as taught by Fagioli. By identifying the 
active window on the GUI, the remote control system of Paroz and Bickmore would be 
able to select that web page corresponding to the window to be displayed on a mobile 
device. This would allow the mobile device to display information the user would most 
want to see. 
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3-2. Regarding claims 2, 19, 39, and 49, Paroz, Bickmore, and Fagioli teach the 
invention substantially as claimed. Paroz further teaches the claim of the system further 
comprising operation execution means for causing said device to be operated to 
execute predetermined processing corresponding to user's operation conducted using 
said operation side terminal, by disclosing that a command executor receives messages 
from the second computing device and translates them into commands executed on the 
local program [column 10, lines 12-16], 

3-3. Regarding claims 3, 20, 37, and 50, Paroz, Bickmore, and Fagioli teach the 
invention substantially as claimed. Paroz further teaches the claim of the system further 
comprising screen information conversion means for converting data of an analysis 
result obtained by said screen analysis means into data suitable for thetransmission to 
said operation side terminal or for displaying on a screen of said operation side terminal, 
wherein said screen information transmission means transmits data of a result of 
conversion by said screen information conversion means to said operation side terminal 
to display the data on said operation side terminal, by disclosing a window analyzer that 
generates a DHTML page for each layout [column 8, lines 51-65], 



3-4. Regarding claims 4, 21, 38, and 51, Paroz, Bickmore, and Fagioli teach the 
invention substantially as claimed. Paroz further teaches the claim wherein the system 
further comprising conversion rule registration means for registering and recording rules 
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for converting data of an analysis result obtained by said screen analysis means into 
data suitable for the transmission to said operation side terminal or for displaying on the 
screen of said operation side terminal, wherein said screen information conversion 
means converts data of an analysis result obtained by said screen analysis means 
based on the conversion rules recorded in said conversion rule registration means, by 
disclosing the window analyzer can generate the DHTML pages according to 
parameters such as target second computing device, communication type, or target 
browser [column 8, lines 57-65]. 



3-5. Regarding claims 7, 24, 41 , and 54, Paroz, Bickmore, and Fagioli teach the 
invention substantially as claimed. Paroz further teaches the claim of the system 
wherein said device to be operated is an information processing device;having a GUI 
and said screen analysis means detects GUI widgets displayed on the screen of said 
device to be operated to obtain attribute data of said GUI widgets [column 6, glossary; 
figure 5]. 

3-6. Regarding claims 8, 25, 42, and 55, Paroz, Bickmore, and Fagioli teach the 
invention substantially as claimed. Paroz further teaches the claim of the system 
wherein the attribute data of said GUI widgets includes at least one of a kind of said GUI 
widgets, a name of said GUI widgets and a location of aid GUI widgets in the screen, by 



disclosing that the DHTML code that the window analyzer creates for a 



widget consists 



of the static attributes to display, handlers for events received from the local programs, 
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and handlers for events occurring on the second computing device [column 9, lines 22- 
26; figure 3], 

3-7. Regarding claims 9, 26, 43, and 56, Paroz, Bickmore, and Fagioji teach the 
invention substantially as claimed. Paroz further teaches the system wherein said 
conversion rule registration means has registration of rules for converting data of an 
analysis result obtained by said screen analysis means into a web page that can be 
displayed by a web browser and said screen conversion means converts data of an 
analysis result obtained by said screen analysis means into a web page based on said 
rules, by disclosing that the window analyzer generates a DHTML for each layout 
according to certain parameters [column 8, lines 51-65], 

Paroz teaches when a GUI widget is selected, data identifying the GUI widget in 
question is transmitted to said operation execution means, by disclosing [figure 6; 
column 10, lines 26-36], 

Bickmore teaches the use of hyper-links to navigate through pages of the 
Internet, by disclosing [figure 1]. Since the remote computing device can be a mobile 
device, it would have been obvious to one of ordinary skill in the at the time the 
invention was made to use the hyper-link system of Bickmore to navigate through the 
HTML pages of Paroz. This would conserve wireless bandwidth and memory. 



3-8. Regarding claims 12 and 29, Paroz, Bickmore, and Fagioli teach; the invention 
substantially as claimed. Paroz further teaches wherein data of an analysis result 
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obtained by said screen analysis means is data of an XML format, by disclosing that a 
list of events, in XML format, made on the local program are processed by the DHTML 
and sent to the mediator [column 10, lines 45-51]. 



3-9. Regarding claims 13 and 30, Paroz, Bickmore, and Fagioli teach: the invention 
substantially as claimed. Paroz further teaches the system wherein said operation side 
terminal is a portable communication terminal having a character data transmission and 
reception function, by disclosing that the second computing device may; be a mobile 
phone [column 3, lines 29-34]. 

3-10. Regarding claims 14 and 31, Paroz, Bickmore, and Fagioli teach; the invention 
substantially as claimed. Paroz further teaches the system wherein said communication 
network is the Internet [column 7, lines 32-38]. 



3-11. Regarding claims 15, 32, 45, and 58, Paroz, Bickmore, and Fagioli teach the 
invention substantially as claimed. Paroz further teaches the claim of the system 
wherein said screen analysis means, when moving picture or still picture is displayed on 
the screen of said device to be operated, detects the display in question to extract 
picture data of the display in question, and said screen information transmission means 
transmits said picture data extracted by said screen analysis means to said operation 
side terminal to display the data on said operation side terminal, by disclosing that the 
visual status monitor may obtain embedded images in the GUI, which are transmitted to 
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the mediator for display on the second computing device [column 9, lines 45-67; column 
10, lines 1-11]. Screen analysis is performed only on objects shown on the first 
computing device's user interface. If no moving or still images were shown on the first 
computing device's user interface, screen analysis would not perform any picture data 
extraction because there would be none to perform. 



3-12. Regarding claims 16, 33, 46, and 59, Paroz, Bickmore, and Fagioli teach the 
invention substantially as claimed. Paroz further teaches the claim of the system 
wherein said screen analysis means, when moving picture or still picture is displayed on 
the screen of said device to be operated, detects the display in question to extract 
picture data of the display in question, and screen information conversion means 
converts said picture data extracted by said screen analysis means into data suitable for 
the transmission to said operation side terminal or for displaying on the screen of said 
operation side terminal, by disclosing that the visual status monitor may obtain 
embedded images in the GUI, which are transmitted to the mediator for display on the 
second computing device [column 9, lines 45-67; column 10, lines 1-11]. The DHTML 
pages may be customized according to parameters such as target second computing 
device [column 8, lines 51-65]. Screen analysis is performed only on objects shown on 
the first computing device's user interface. If no moving or still images were shown on 
the first computing device's user interface, screen analysis would not perform any 
picture data extraction because there would be none to perform. 
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Patent No. 6,857,102), Fagioli (U.S. Patent No. 6,710,790), and Martin, 
Patent No 6,610,105). 



Jr. et al (US 



4-1. Regarding claims 5, 22, 39, and 52, Paroz, Bickmore, and Fagioji teach the 
invention substantially as claimed. Paroz, Bickmore, and Fagioli do not expressly teach 
the system further comprising device characteristic detection means for detecting 
device characteristics of said operation side terminal, wherein said screen information 
conversion means converts data of an analysis result obtained by said screen analysis 
means based on device characteristics detected by said device characteristic detection 
means and conversion rules recorded in conversion rule registration means. 

Martin, Jr. teaches a remote access system, similar to that of Paroz, Bickmore, 
and Fagioli wherein Martin, Jr. further teaches techniques to facilitate participation of a 
mobile device in accessing resources over a data network [column 2, lines 34-36], 
Martin, Jr. teaches receiving a request for a web page from a requestor, determining 
device characteristics of a mobile device used by the requestor, retrieving menu 
information based on the device characteristics of the mobile device used by the 
requester, producing the navigation aid based on the menu information; the navigation 
aid being in a format suitable for the mobile device, and forwarding the web page 
including the navigation aid to the requester [column 2, lines 49-56]. 
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It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include in the remote control system of Paroz, Bickmore, and 
Fagioli, the technique taught by Martin, Jr. for producing a navigation aid on a web page 
for a mobile device by detecting device characteristics and converting the navigation aid 
into a format suitable for the mobile device. This would allow many different mobile 
devices with different operating environments to access the GUI web page representing 
the GUI of a computer, and thus, provide remote access to a computer using the mobile 
device, as suggested by Martin, Jr. 

4-2. Regarding claims 17, 34, 47, 60, Paroz, Bickmore, and Fagioli teach the 
invention substantially as claimed. Paroz, Bickmore, and Fagioli do not expressly teach 
the system wherein device characteristics of said operation side terminal detected by 
said device characteristic detection means include at least one of an inherent ID of said 
operation side terminal, a kind of machine, a processing rate, a communication rate, 
communication costs per unit volume of data, communication costs periunit time, a data 
format that can be displayed on a screen, a playable sound data format, executable 
program and script formats, a volume of data receivable at one time, a color that can be 
displayed, a character font that can be displayed, a screen resolution, a physical length 
of a screen in a vertical direction and a lateral direction and the number; of characters 
that can be displayed within one screen in the vertical direction and lateral direction 
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Martin, Jr. teaches that the device information obtained includes one or more of 
device characteristics, rendering context for the menu, and user identify and/or browser 
type utilized by the requesting device [column 11, lines 5-9]. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include in the remote access system of Paroz, Bickmore, and 
Fagioli, device characteristics of the device taught by Martin, Jr. in order to be able to 
determine what kind of device is being used to remotely access a computer so as to 



more accurately produce the interface for remote access, as suggested 



by Martin, Jr. 



being 



5. Claims 6, 23, 40, and 53 are rejected under 35 U.S.C. 103(a) as; 
unpatentable over Paroz et al (U.S. Patent No. 6,587,125), Bickmore et al (U.S. Patent 
No. 6,857,102), Fagioli (U.S. Patent No. 6,710,790), and Suzuki et al (U.S. Patent No. 
6,003,067). 

5-1. Regarding claims 6, 23, 40, and 53, Paroz, Bickmore, and Fagioli teach the 
invention substantially as claimed. Paroz teaches the claim of the system further 
comprising screen change detection means for detecting a change in display of the 
screen of said device to be operated, wherein at a time when a change occurred in 
display of the screen is completed, said screen analysis means analyzes the current 
screen of said device to be operated, by disclosing a visual status monitor that monitors 
GUI events generated by the local program and updates the widgets running on the 
second computing device [column 8, lines 66-67; column 9, lines 1-2]. Paroz further 
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teaches that DHTML pages can be customized for properties such as refresh rate 
[column 8, lines 57-60]. 

Paroz, Bickmore, and Fagioli do not expressly teach wherein said screen change 
detection means determines that the change that occurred in the display of the screen 
is completed when no screen change is detected in the display of the screen for more 
than a predetermined time period. Suzuki teaches a data transmission processing 
system for reducing the transmission data amount and quickly conducting the displaying 
operation of screen data by transmitting from a second information processing 
apparatus to a first information processing apparatus the screen data [column 2, lines 
21-29]. The invention of Suzuki detects whether or not there is a change in the screen 
after a predetermined period of time and if so, transmits the changed screen data to the 
first information processing apparatus [column 10, lines 1-53]. It would have been 
obvious to one of ordinary skill in the art at the time the invention was made, to include 
in the remote control system of Paroz, Bickmore, and Fagioli, detecting jthat the change 
that occurred in the display of the screen is completed when no screen change is 
detected in the display of the screen for more than a predetermined time period, as 
taught by Suzuki, and performing the function of analyzing the screen subsequently. 
This would ensure that a change in the screen of the device to be operated is noticed so 
that the display on the operation side terminal can be updated accordingly. 



6. Claims 10-11, 27-28, 44, and 57 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Paroz et al (U.S. Patent No. 6,587,125), Bickmore et al (U.S. Patent 
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No. 6,857,102), Fagioli (U.S. Patent No. 6,710,790), and Rosenblatt et al (US Patent No 
6,263,363). 

6-1. Regarding claims 10, 27, 44, and 57, Paroz, Bickmore, and Fagioli teach the 
invention substantially as claimed. Paroz, Bickmore, and Fagioli do not expressly teach 
the system wherein said conversion rule registration means has registration of rules for 
converting data of an analysis result obtained by said screen analysis means into a text 
of electronic mail. Rosenblatt teaches a similar invention for remotely accessing a 
computer over the Internet. Rosenblatt teaches that changes made to the virtual copy 
(the device to be operated) during a session remotely controlled from the remote access 
client computer are sent to the home computer to synchronize the home computer with 
its virtual copy. The process can be undertaken by several methods according to user 
selection, including emailing the changes to the home computer (or elsewhere as 
designated by the user), selecting where the updates will be stored on the home 
computer, and selecting how file conflicts are handled [column 9, lines 42-52]. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include in the remote access system of Paroz, Bickmore, and 
Fagioli, e-mailing the changes made to the device to be operated as taught by 
Rosenblatt. This would allow the user to keep track of what operations have been 
performed remotely. 
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6-2. Regarding claims 1 1 and 28, Paroz, Bickmore, Fagioli, and Rosenblatt teach the 
invention substantially as claimed. Rosenblatt further teaches the system wherein 
operation execution means receives electronic mail including information of user's 
operation conducted using said operation side terminal, analyzes the received 
electronic mail to identify the user's operation, and executes the operation in question, 
by disclosing that changes made to the virtual copy (the device to be operated) during a 
session remotely controlled from the remote access client computer are sent to the 
home computer to synchronize the home computer with its virtual copy. The process 
can be undertaken by several methods according to user selection, including emailing 
the changes to the home computer (or elsewhere as designated by the user), selecting 
where the updates will be stored on the home computer, and selecting how file conflicts 
are handled [column 9, lines 42-52], In order to synchronize the home computer with its 
virtual copy, the home computer must identify the changes and perform- all the 
operations carried out by the remote computer, which are listed in the e-mail. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include in the remote access system of Paroz, Bickmore, and 
Fagioli, the method of using e-mail to communicate the commands and changes made 
to the device to be operated as taught by Rosenblatt. This would allow a user to update 
a remote device by using standard e-mail protocols. 



Response to Arguments 
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7. Regarding claims 1,18, 35, and 48, the Applicants allege that Paroz et al (U.S. 
Patent No. 6,587,125), Bickmore et al (U.S. Patent No. 6,857,102), and Fagioli (U.S. 
Patent No. 6,710,790) as described in the previous Office action, does not explicitly 
teach that the screen analysis means and the screen information transmission means 
are provided on said device to be operated, as recited in claim 18, and amended in 
claims 1 , 35, and 48. Bickmore, however, teaches that the document re-authoring 
system can be implemented on a programmed general purpose computer as well as on 
a special purpose computer, a programmed microprocessor or microcontroller and 
peripheral integrated circuit elements, an ASIC or other integrated circuit, a digital signal 
processor, a hardwired electronic or logic circuit such as a discrete element circuit, a 
programmable logic device such as PLD, PLA, FPGA, or PAL, or the like. In general, 
any device capable of implementing a finite state machine can be used ito implement 
the automatic document re-authoring system [column 27, lines 26-48]. Thus, it would 
have been obvious for one of ordinary skill in the art to implement the re-authoring 
system on the device to be operated in order to reduce the processing load on the client 
device and eliminate the need for an additional device leading to additional overhead 
that may be needed. Consequently, and given the broadest, most reasonable 
interpretation of their claim language, Paroz, Bickmore, and Fagioli are still considered 
to teach claims 1 , 1 8, 35, and 48. 

Applicant's arguments, see [page 21, paragraph 6] of Applicant's remarks, filed 
4/1 1/06, with respect to the rejections of claims 6, 23, 40, and 53 under 35 USC § 103 
have been fully considered and are persuasive. Therefore, the rejection has been 
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withdrawn. However, upon further consideration, a new ground of rejection is made in 
view of Paroz et al, Bickmore et al, Fagioli, and Suzuki et al (U.S. Patent No. 
6,003,067). See section 5-1 of this Office action. 

Regarding claims 7-9, the Applicants allege that Paroz et al, Bickmore et al, and 
Fagioli as described in the previous Office action, does not explicitly teach providing 
GUI widget information along with the x, y, z coordinate position of an active window. 
Claims 7 and 8, however, only recite obtaining attribute data of GUI widgets, the 
attribute data including at least one of a kind of said GUI widgets, a name of said GUI 
widgets, and a location of said GUI widgets in the screen. Claim 9 only recites a hyper- 
link corresponding to GUI widgets and identifying GUI widgets corresponding to the 
hyper-link. Nothing in claims 7, 8, or 9 recites providing x, y, z coordinate position of an 
active window. Therefore, Paroz, Bickmore, and Fagioli are still considered to teach 
claims 7-9. 

Regarding claims 15 and 16, the Applicants allege that Paroz et al, Bickmore et 
al, and Fagioli as described in the previous Office action, does not explicitly teach that 
the screen analysis means does not perform any picture data extraction when moving 
picture or still picture is not displayed on the screen of said device to be: operated. 
Paroz, however, teaches that an equivalent user interface of the first computing device 
is sent to the second computing device [column 3, lines 55-62]. Thus, screen analysis is 
performed only on objects shown on the first computing device's user iriterface. If no 
moving or still images were shown on the first computing device's user interface, screen 
analysis would not perform any picture data extraction because there would be none to 
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perform. Consequently, and given the broadest, most reasonable interpretation of their 
claim language, Paroz, Bickmore, and Fagioli are still considered to teach claims 15 and 
16. 

Claims 1,18, 35, and 48 are rejected as discussed above and consequently, 
claims 2-17, 19-34, 36-47, and 49-60 which recite all the limitations of independent 
claims 1,18, 35, and 48 remain rejected. 



Conclusion 

8. A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 



9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Alvin H. Tan whose telephone number is 571-272-8595 . 
The examiner can normally be reached on Mon-Thu 9:30-7 and alternating Fridays 
9:30-7. 



Application/Control Number: 09/910,778 



Page 20 



Art Unit: 2173 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kristine Kincaid can be reached on 571-272-4063. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.usptp.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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